Image capturing device capable of blending images and image processing method for blending images thereof

ABSTRACT

An image capturing device and an image processing method are provided. The present method includes following steps. A first image and a second image are captured with a first focal length and a second focal length correspondingly. The motion corrected second image is produced by performing geometric correction procedure on the second image. A gradient operation is performed on each of the pixels of the first image to obtain a plurality of first gradients, and the gradient operation is performed on each of the pixels of the motion corrected second image to obtain a plurality of second gradients. The first gradients and the second gradients are compared and a first parameter map is generated according to the comparison results. A blending image is produced in according with the first parameter map and the first image, and an output image is produced at least in according with the blending image.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the priority benefit of Taiwan applicationserial no. 102104649, filed on Feb. 6, 2013, and Taiwan applicationserial no. 102122757, filed on Jun. 26, 2013. The entirety of each ofthe above-mentioned patent application is hereby incorporated byreference herein and made a part of this specification.

BACKGROUND

1. Technical Field

The invention relates to an image capturing device and an imageprocessing method thereof. Particularly, the invention relates to animage processing method capable of blending images by calculating pixelgradients.

2. Related Art

Along with development of optical technology, digital cameras capable ofadjusting aperture, shutter and even changing lenses are widely used,and functions of the digital camera tend to be diversified. Besides thatthe digital camera is required to provide good imaging quality, accuracyand speed of a focusing technique are also factors considered by aconsumer when the consumer purchases the digital camera. However,regarding an existing optical system, since different objects havedifferent distances in a three-dimensional scene, an entirely clear fulldepth of field (DOF) image cannot be obtained in a single imagecapturing process. Namely, limited by a lens optical characteristic,when the digital camera is used to capture image, only one of depths isselected for focusing, so that objects of other depths in the image areless clear.

A conventional method of producing a full DOF image is to combine aplurality of images captured according to different capturingconditions. Different images of one scene are captured by changing oneor a plurality of parameters in the capturing condition, and theseimages are combined into one clear image according to a claritydetermination method. The technique of capturing images according to theabove different capturing conditions to produce the full DOF imagerelies on a fixed image capturing device. Generally, a user usually usesa stable tripod to fix the image capturing device, so as to ensure thatthe captured images have none obvious geometric distortion therebetween. Moreover, during the image capturing process, movement of anyobject in the scene to be captured has to be avoided as well.

On the other hand, when the camera is used to capture images, in orderto highlight a theme of the captured image, an image capturing techniqueof “bokeh” is generally adopted. The so-called “bokeh” refers to that ina captured image with a shallow DOF, the part of image outside the DOFgradually produces a loose blur effect. Generally, a bokeh effectproduced by the camera lens is limited. In order to obtain a betterbokeh effect, important conditions of large aperture and long focallength have to be satisfied. In other words, in order to achieve abetter bokeh effect, a large aperture lens is used to strengthen blur ofdistant objects, so as to highlight the clear theme from the background.However, the large aperture lens has a large volume and high cost, whichis not suitable for the general consumable cameras.

Therefore, the conventional method for producing the full DOF image orthe dokeh image is liable to cause a problem that the processed imagehas a discontinuous DOF or unnatural result. Moreover, limitation on theimage capturing operation is inconvenient to the user, for example, along average total image capturing time or a complicated operatingprocess, which even leads to an unsatisfactory final image.

SUMMARY

Accordingly, the invention is directed to an image capturing device andan image processing method thereof, by which a main object of an imageis determined according to images captured with different focal lengths,so as to generate an image with a clear main object and a natural bokeheffect. On the other hand, in the image processing method, a ghostphenomenon occurred when generating a full depth of field (DOF) image isavoided according to the images captured with different focal lengths.

The invention provides an image processing method, which is adapted toan image capturing device. The image processing method includesfollowing steps. A first image and a second image are captured with afirst focal length and a second focal length, where the first focallength is focused on at least one main object. A geometric calibrationprocedure is performed on the second image to produce the motioncalibrated second image. A gradient operation is performed on each pixelof the first image to produce a plurality of first gradients, and thegradient operation is performed on each pixel of the motion calibratedsecond image to produce a plurality of second gradients. Each of thefirst gradients and the corresponding second gradient are compared togenerate a plurality of first pixel comparison results, and a firstparameter map is generated according to the first pixel comparisonresults. A blending image is produced according to the first parametermap and the first image, and an output image is produced at least inaccordance with the blending image.

In an embodiment of the invention, the step of producing the outputimage at least in accordance with the blending image includes followingsteps. A third image is captured with a third focal length. Thegeometric calibration procedure is performed on the third image toproduce the motion calibrated third image. The gradient operation isperformed on each pixel of the blending image to generate a plurality ofthird gradients, and the gradient operation is performed on each pixelof the motion calibrated third image to generate a plurality of fourthgradients. Each of the third gradients and the corresponding fourthgradient are compared to generate a plurality of second pixel comparisonresults, and a second parameter map is generated according to the secondpixel comparison results. The motion calibrated third image and theblending image are blended according to the second parameter map toproduce the output image.

In an embodiment of the invention, the step of performing the geometriccalibration procedure on the second image to produce the motioncalibrated second image includes following steps. A motion amountestimation is performed on the first image and the second image tocalculate a homography matrix. A geometric affine transformation isperformed on the second image according to the homography matrix, so asto obtain the motion calibrated second image.

In an embodiment of the invention, the step of comparing the each of thefirst gradients and the corresponding second gradient to generate thefirst pixel comparison results, and generating the first parameter mapaccording to the first pixel comparison results includes followingsteps. The second gradients are divided by the corresponding firstgradients to generate a plurality of gradient comparison values. Aplurality of parameters are generated according to the gradientcomparison values, and the parameters are recorded as the parameter map.

In an embodiment of the invention, the step of generating the parametersaccording to the gradient comparison values includes following steps. Itis determined whether the gradient comparison values are greater than afirst gradient threshold. The parameters corresponding to the gradientcomparison values are set to a first value when the gradient comparisonvalues are greater than the first gradient threshold.

In an embodiment of the invention, the step of generating the parametersaccording to the gradient comparison values includes following steps. Itis determined whether the gradient comparison values are greater than asecond gradient threshold when the gradient comparison values are notgreater than the first gradient threshold. The parameters correspondingto the gradient comparison values are set to a second value when thegradient comparison values are greater than the second gradientthreshold. The parameters corresponding to the gradient comparisonvalues are set to a third value when the gradient comparison values arenot greater than the second gradient threshold, where the first gradientthreshold is greater than the second gradient threshold.

In an embodiment of the invention, the step of producing the blendingimage according to the first parameter map and the first image includesfollowing steps. A blur procedure is performed on the first image togenerate a blur image. The first image and the blur image are blendedaccording to the first parameter map to produce a main object clearimage.

In an embodiment of the invention, the step of blending the first imageand the blur image according to the first parameter map to produce themain object clear image includes following steps. It is determinedwhether the parameters are greater than a first blending threshold.Pixels of the blur image corresponding to the parameters are obtained toserve as pixels of the main object clear image when the parameters aregreater than the first blending threshold. It is determined whether theparameters are greater than a second blending threshold when theparameters are not greater than the first blending threshold. Pixels ofthe main object clear image are calculated according to the parameterswhen the parameters are greater than the second blending threshold.Pixels of the first image corresponding to the parameters are obtainedto serve as pixels of the main object clear image when the parametersare not greater than the second blending threshold, where the firstblending threshold is greater than the second blending threshold.

In an embodiment of the invention, the step of producing the blendingimage according to the first parameter map and the first image includesfollowing steps. A plurality of sums of absolute differencescorresponding to each pixel is calculated according to a pixel value ofeach of the pixels in the first image and the second image, and theparameters in the first parameter map are adjusted according to the sumsof absolute differences. The first image and the motion calibratedsecond image are blended according to the adjusted first parameter mapto generate a full depth of field image.

In an embodiment of the invention, the step of calculating the sums ofabsolute differences corresponding to each pixel according to the pixelvalue of each of the pixels in the first image and the second image andadjusting the parameters in the first parameter map according to thesums of absolute differences includes following steps. A weightingfactor of each of the parameters is determined according to the sums ofabsolute differences when the sums of absolute differences are greaterthan a motion threshold, and the parameters are adjusted according tothe weighting factor, where each of the parameters decreases as thecorresponding sum of absolute difference increases.

In an embodiment of the invention, the step of blending the first imageand the motion calibrated second image according to the weightingfactor-adjusted first parameter map to generate the full depth of fieldimage includes following steps. It is determined whether the parametersare greater than a first blending threshold. Pixels of the motioncalibrated second image corresponding to the parameters are obtained toserve as pixels of the full depth of field image when the parameters aregreater than the first blending threshold. It is determined whether theparameters are greater than a second blending threshold when theparameters are not greater than the first blending threshold. Pixels ofthe full depth of field image are calculated according to the parameterswhen the parameters are greater than the second blending threshold.Pixels of the first image corresponding to the parameters are obtainedto serve as pixels of the full depth of field image when the parametersare not greater than the second blending threshold, where the firstblending threshold is greater than the second blending threshold.

According to another aspect, the invention provides an image capturingdevice including an image capturing module, a motion calibrating module,a gradient calculating module, a map generating module and an imageblending module. The image capturing module captures a first image witha first focal length and captures a second image with a second focallength, where the first focal length is focused on at least one mainobject. The motion calibrating module performs a geometric calibrationprocedure on the second image to produce the motion calibrated secondimage. The gradient calculating module performs a gradient operation oneach pixel of the first image to produce a plurality of first gradients,and performs the gradient operation on each pixel of the motioncalibrated second image to produce a plurality of second gradients. Themap generating module compares each of the first gradients and thecorresponding second gradient to generate a plurality of first pixelcomparison results, and generates a first parameter map according to thefirst pixel comparison results. The image blending module produces ablending image according to the first parameter map and the first image,and produces an output image at least in accordance with the blendingimage.

According to the above descriptions, based on a characteristic thatdifferent focal lengths lead to different images, a same scene iscaptured with different focal lengths, and gradient differences of eachpixel between the images are compared to generate the parameter map.According to the information of the parameter map, a clear full depth offield image or a bokeh image with a clear main object and a blurrybackground is generated, so as to achieve a better full depth of fieldeffect or a bokeh effect.

In order to make the aforementioned and other features and advantages ofthe invention comprehensible, several exemplary embodiments accompaniedwith figures are described in detail below.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings are included to provide a furtherunderstanding of the invention, and are incorporated in and constitute apart of this specification. The drawings illustrate embodiments of theinvention and, together with the description, serve to explain theprinciples of the invention.

FIG. 1 is a functional block diagram of an image capturing deviceaccording to an embodiment of the invention.

FIG. 2 is a flowchart illustrating an image processing method accordingto an embodiment of the invention.

FIG. 3 is a schematic diagram of an image processing method according toanother embodiment of the invention.

FIG. 4 is a block diagram of an image capturing device according tostill another embodiment of the invention.

FIG. 5 is a flowchart illustrating an image processing method accordingto still another embodiment of the invention.

FIG. 6 is a flowchart illustrating detailed steps of a step S550 of FIG.5 according to still another embodiment of the invention.

FIG. 7 is a flowchart illustrating detailed steps of a step S560 of FIG.5 according to still another embodiment of the invention.

FIG. 8 is a block diagram of an image capturing device according tostill another embodiment of the invention.

FIG. 9A is a schematic diagram of pixel blocks according to stillanother embodiment of the invention.

FIG. 9B is a schematic diagram illustrating a relationship between a sumof absolute differences and a weighting factor according to stillanother embodiment of the invention.

DETAILED DESCRIPTION OF DISCLOSED EMBODIMENTS

The invention provides a method for generating a bokeh image and a fulldepth of field (DOF) image according to a plurality of images capturedwith different focal lengths. At least one main object is focused andcaptured, and then the same scene is captured with another focal length.A parameter map is generated by comparing pixel gradients of the twoimages, and the main object in the image is accordingly determined, soas to produce an image with a bokeh effect. On the other hand, theparameter map used for image blending is generated by comparing pixelgradients of at least two images, so as to produce the full DOF image.In order to clearly describe the content of the invention, referencewill now be made in detail to the present preferred embodiments of theinvention.

FIG. 1 is a functional block diagram of an image capturing deviceaccording to an embodiment of the invention. Referring to FIG. 1, theimage capturing device 100 of the present embodiment is, for example, adigital camera, a monocular camera, a digital video camera or a smartphone, a tablet PC, a head mounted display, etc. that have an imagecapturing function, though the invention is not limited thereto. Theimage capturing device 100 includes an image capturing module 110, animage calibrating module 120, a gradient calculating module 130, a mapgenerating module 140 and an image blending module 150.

The image capturing module 110 includes a zoom lens and a photosensitiveelement, the photosensitive element is, for example, a charge coupleddevice (CCD), a complementary metal-oxide semiconductor (CMOS) or otherdevice, the image capturing module 110 may further include an aperture,etc., which is not limited by the invention. The image capturing module110 may capture different images with different focal lengths.

On the other hand, the image calibrating module 120, the gradientcalculating module 130, the map generating module 140 and the imageblending module 150 can be implemented by software, hardware or acombination thereof, which is not limited by the invention. The softwareis, for example, source codes, operating system, application software ordriving program, etc. The hardware is, for example, a central processingunit (CPU), or other programmable general-purpose or special-purposemicroprocessor.

FIG. 2 is a flowchart illustrating an image processing method accordingto an embodiment of the invention. The method of the present embodimentis adapted to the image processing device 100 of FIG. 1, detailed stepsof the image processing method are described below with reference ofvarious modules of the image capturing device 100.

First, in step S210, the image capturing module 110 captures a firstimage with a first focal length and captures a second image with asecond focal length, where the first focal length is focused on at leastone main object. Namely, the image capturing module 110 captures twoimages with two different focal lengths. Under a same condition, theimages captured with different focal lengths are different. In detail,regarding the first image focusing on the main object, the main objectin the image is the clearest part.

In step S220, the image calibrating module 120 performs a geometriccalibration procedure on the second image to produce the motioncalibrated second image. Since the first image and the second image areobtained by continuously capturing images of the same scene, images ofdifferent angles are probably captured due to shaking or moving of thecamera, i.e. the first image and the second image may have adisplacement. Therefore, the image calibrating module 120 performs thegeometric calibration procedure on the second image. In other words,after the geometric calibration procedure, a position of a startingpixel of the motion calibrated second image can be the same to aposition of a starting pixel of the first image.

In step S230, the gradient calculating module 130 performs a gradientoperation on each pixel of the first image to produce a plurality offirst gradients, and performs the gradient operation on each pixel ofthe motion calibrated second image to produce a plurality of secondgradients. Namely, each pixel in the first image has the first gradient,and each pixel in the motion calibrated second image has the secondgradient.

In step S240, the map generating module 140 compares each of the firstgradients and the corresponding second gradient to generate a pluralityof first pixel comparison results, and generates a first parameter mapaccording to the first pixel comparison results. In brief, the mapgenerating module 140 compares the gradients of the pixels located atthe same position, and generates a pixel comparison result for eachpixel position.

In step S250, the image blending module 150 produces a blending imageaccording to the first parameter map and the first image, and producesan output image at least in accordance with the blending image. Indetail, after obtaining the parameter map, the image capturing device100 blends the first image and the image processed with other imageprocessing according to the parameter map, so as produce the blendingimage. Moreover, the image capturing device 100 can also blend the firstimage and the second image according to the parameter map, so as toproduce the blending image.

It should be noticed that although two images captured with two focallengths are taken as an example for descriptions in the aforementionedembodiment, the invention is not limited thereto. According to an actualapplication, a plurality of images captured with a plurality of focallengths can be obtained to produce the final output image. For example,since the images captured with different focal lengths respectively havedifferent clear image parts, a clear full DOF image can be obtainedaccording to the images captured with different focal lengths. Moreover,the image processing method of the invention can also produce an outputimage in which only the main object is clear according to three imagesrespectively focused on the main object, the background and theforeground. An embodiment is provided below for detailed descriptions.

FIG. 3 is a schematic diagram of an image processing method according toanother embodiment of the invention. In the present embodiment, theimage capturing module 110 respectively captures a first image Img1 anda second image Img2 with a first focal length and a second focal length.Then, the same to the aforementioned embodiment, a blending image Img_bis generated based on processing of the image calibrating module 120,the gradient calculating module 130, the map generating module 140 andthe image blending module 150, which is not repeated. It should benoticed that in the aforementioned embodiment, the image blending module150 can take the blending image Img_b as the final output image, thoughin the present embodiment, the blending image Img_b is further blendedwith another image to produce a final output image Img_F. In detail, asthat shown in FIG. 3, the image capturing module 110 captures a thirdimage Img3 with a third focal length. The image calibrating module 120performs the geometric calibration procedure on the third image Img3 toproduce the motion calibrated third image Img3.

Thereafter, the gradient calculating module performs a gradientoperation on each pixel of the blending image Img_b to generate aplurality of third gradients, and performs the gradient operation oneach pixel of the motion calibrated third image Img3 to generate aplurality of fourth gradients. The map generating module 140 compareseach of the third gradients and the corresponding fourth gradient togenerate a plurality of second pixel comparison results, and generates asecond parameter map according to the second pixel comparison results.The second parameter map is obtained by calculating gradients of theblending image Img_b and the third image Img3, and the internalparameters thereof are different to that of the first parameter mapcalculated according to the first image Img1 and the second image Img2.The image blending module 150 blends the motion calibrated third imageImg3 and the blending image Img_b to produce the output image Img_faccording to the second parameter map. Therefore, the number of imagesblended for obtaining the final output image is not limited by theinvention, which is determined according to an actual applicationrequirement.

However, implementation of the invention is not limited to theaforementioned description, and the content of the aforementionedembodiment can be varied according to an actual requirement. Forexample, in another embodiment of the invention, the image capturingdevice may further include an image blurring module to produce a mainobject clear image having the bokeh effect. Moreover, in still anotherembodiment of the invention, the image capturing device may furtherinclude a map adjusting module to produce a full DOF image having abetter full DOF effect. Embodiments are provided below to describe howthe gradient calculating module, the map generating module and the imageblending module produce the bokeh image and the full DOF image accordingto images captured with different focal lengths.

FIG. 4 is a block diagram of an image capturing device according tostill another embodiment of the invention. The image capturing device400 includes an image capturing module 410, an image calibrating module420, a gradient calculating module 430, a map generating module 440, animage blending module 450 and an image blurring module 460. The imagecapturing module 410, the image calibrating module 420, the gradientcalculating module 430, the map generating module 440 and the imageblending module 450 are similar to the image capturing module 110, theimage calibrating module 120, the gradient calculating module 130, themap generating module 140 and the image blending module 150 of FIG. 1,so that details thereof are not repeated. The embodiment of FIG. 4 canbe deduced according to related descriptions of the embodiments of FIG.1 to FIG. 3.

It should be noticed that different to the image capturing device 100 ofFIG. 1, the image capturing device 400 further include an image blurringmodule 460. The image blurring module 460, for example, adopts aGaussian filter, a bilateral filter or an average filter used forperforming a blur procedure on the first image Img1, which is notlimited by the invention. Moreover, in the present embodiment, it isassumed that the second focal length is focused on the background.

FIG. 5 is a flowchart illustrating an image processing method accordingto still another embodiment of the invention. The image processingmethod of the present embodiment is adapted to the image capturingdevice 400 of FIG. 4, and detailed steps of the present embodiment aredescribed below with reference of various modules of the imageprocessing device 400.

First, in step S510, the image capturing module 410 the image capturingmodule 410 respectively captures the first image Img1 with the firstfocal length and captures the second image Img2 with the second focallength, where the first focal length is focused on at least one mainobject, and the second focal length is focused on the background. In thefirst image Img1 captured by focusing on the main object, the mainobject is clear and the background is blurry. Compared to the firstimage Img1, in the second image Img2 captured by focusing on thebackground, the background is clear. Then, in step S520, the imageblurring module 460 performs the blur procedure on the first image Img1to produce a blur image Img1_blur.

In step S530, the image calibrating module 430 performs a geometriccalibration processing on the second image Img2 to produce the motioncalibrated second image Img2_cal. In detail, the image calibratingmodule 430 performs a motion amount estimation on the first image Img1and the second image Img2 to calculate a homography matrix. Then, theimage calibrating module 430 performs a geometric affine transformationon the second image Img2 according to the homography matrix, so as toobtain the transformed motion calibrated second image Img2_cal. In thisway, a position of a starting pixel of a main object area in the firstimage Img1 can be the same to a position of a starting pixel of the mainobject area in the motion calibrated second image Img2_cal.

Then, in step S540, the gradient calculating module 440 performs agradient operation on each pixel of the first image Img1 to produce aplurality of first gradients G1, and performs the gradient operation oneach pixel of the motion calibrated second image Img2_cal to produce aplurality of second gradients G2. The gradient operation can be ahorizontal gradient operation, a vertical gradient operation or dualdiagonal gradient operations, which is not limited by the invention.Namely, the first gradient and the second gradient can be a horizontalgradient, a vertical gradient or dual diagonal gradients according tothe method of the gradient operation. The horizontal gradient is a sumof absolute grayscale differences between the pixel and two adjacentpixels in the horizontal direction. The vertical gradient is a sum ofabsolute grayscale differences between the pixel and two adjacent pixelsin the vertical direction. The diagonal gradient is a sum of absolutegrayscale differences between the pixel and pixels in the diagonaldirection.

It should be noticed that in the present embodiment, since the firstimage Img1 is captured by focusing on the main object, compared to themotion calibrated image Img2_cal, the main object in the first imageImg1 is clearer. Namely, the gradient of the pixel in the main objectarea of the first image Img1 is greater than the gradient of the pixellocated at the same position in the motion calibrated second imageImg2_cal. Conversely, since the motion calibrated second image Img2_calis captured by focusing on the background, the gradient of the pixel inthe background area of the first image Img1 is smaller than the gradientof the pixel located at the same position in the motion calibratedsecond image Img2_cal.

Therefore, in step S550, the map generating module 440 compares each ofthe first gradients G1 and the corresponding second gradient G2 togenerate a plurality of comparison results, and generates a parametermap according to the comparison results. It should be noticed that inthe present embodiment, the parameter map is referred to as a bokeh mapbokeh_map. In detail, the map generating module 440 compares thegradients of the pixels located at each same position in the first imageImg1 and the motion calibrated second image Img2_cal. Then, based on therelationship between the gradients of each pixel in the first image Img1and the motion calibrated second image Img2_cal, it is determinedwhether each pixel in the first image Img1 is located in the main objectarea or the background area according to a comparison result. The mapgenerating module 440 generates the bokeh map bokeh_map according to thecomparison result of the gradients of each pixel in the first image Img1and the motion calibrated second image Img2_cal. In other words, thebokeh map bokeh_map carries comparison result information of thegradients of the pixels located at the same position in the first imageImg1 and the motion calibrated second image Img2_cal.

Finally, in step S560, the image blending module 450 blends the firstimage Img1 and the blur image Img1_blur according to the bokeh mapbokeh_map to produce a main object clear image Img1_bokeh. Therefore,the second image Img2 is used for producing the bokeh map bokeh_map, andthe image blending module 450 blends the first image Img1 and the blurimage Img1_blur according to the bokeh map bokeh_map to produce the mainobject clear image Img1_bokeh having the bokeh effect. In this way, thebokeh image with clear main object area and blurry background area isgenerated.

Moreover, how the map generating module 440 generates the bokeh mapbokeh_map according to a comparison result of each of the firstgradients G1 and the corresponding second gradient G2 is described indetail below. FIG. 6 is a flowchart illustrating detailed steps of thestep S550 of FIG. 5 according to still another embodiment of theinvention. Referring to FIG. 4 and FIG. 6, in step S610, the mapgenerating module 440 divides the second gradients G2 by thecorresponding first gradients G1 to generate a plurality of gradientcomparison values. In step S620, the map generating module 440 generatesa plurality of parameters according to the gradient comparison values,and records the parameters to the bokeh map bokeh_map. For example, ifthe first image Img1 and the motion calibrated second image Img2_calrespectively have 1024*768 pixels, 1024*768 gradient comparison valuesare generated through operation of the map generating module 440, andthe bokeh map bokeh_map contains 1024*768 parameters. Here, the stepS620 can be implemented by steps S621-S625.

The map generating module 440 determines whether the gradient comparisonvalue of each position is greater than a first gradient threshold (stepS621). If the gradient comparison value is greater than the firstgradient threshold, the map generating module 440 sets the parametercorresponding to the gradient comparison value to a first value (stepS622), and the first value is referred to as a bokeh background value.In other words, if the gradient comparison value is greater than thefirst gradient threshold, it represents that the pixel of such positionis located in the background area. If the gradient comparison value isnot greater than the first gradient threshold, the map generating module440 determines whether the gradient comparison value is greater than asecond gradient threshold (step S623). If the gradient comparison valueis greater than the second gradient threshold, the map generating module440 sets the parameter corresponding to the gradient comparison value toa second value (step S624), and the second value is referred to as abokeh edge value. In brief, if the gradient comparison value is betweenthe second gradient threshold and the first gradient threshold, itrepresents that the pixel of such position is located in an edge areaconnected between the main object area and the background area. If thegradient comparison value is not greater than the second gradientthreshold, the map generating module 440 sets the parametercorresponding to the gradient comparison value to a third value (stepS625), and the third value is referred to as a bokeh main object value,i.e. the pixel of such position is located in the main object area. Itshould be noticed that the bokeh edge value is between the bokehbackground value and the bokeh main object value, and the first gradientthreshold is greater than the second gradient threshold, and the firstgradient threshold and the second gradient threshold are determinedaccording to an actual requirement, which is not limited by theinvention.

For example, it is assumed that the map generating module 440 sets theparameters to be between 0 and 255, the map generating module 440 cangenerate the bokeh map bokeh_map by using following pseudo code (1):

${if}\mspace{11mu}\left( {\frac{Gra2}{Gra1} > {{TH}\; 1}} \right)$ (1)Map = 255${{else}{if}}\mspace{11mu}\left( {\frac{Gra2}{Gra1} > {{TH}\; 2}} \right)$${Map} = {\frac{\frac{{Gra}\; 2}{{Gra}\; 1} - {TH2}}{{{TH}\; 1} - {{TH}\; 2}} \times 255}$else Map = 0In the present exemplary embodiment, the bokeh background value is 255,the bokeh main object value is 0, and the bokeh edge value can becalculated according to a ratio between the first gradient threshold andthe second gradient threshold and a ratio between the second gradientand the first gradient. Gra2 is the second gradient, Gra1 is the firstgradient, TH1 is the first gradient threshold, TH2 is the secondgradient threshold, and Map is a plurality of parameters in the bokehmap bokeh_map.

Moreover, it is described in detail below how the image blending module450 generates the main object clear image Img1_bokeh by using the bokehmap bokeh_map. FIG. 7 is a flowchart illustrating detailed steps of thestep S560 of FIG. 5 according to an exemplary embodiment of theinvention. Referring to FIG. 4 and FIG. 7, it should be noticed that thepixel of each position in the first image Img1 may correspond to each ofthe parameters in the bokeh map bokeh_map. In step S710, the imageblending module 450 determines whether each of the parameters is greaterthan a first blending threshold. If the parameters are greater than thefirst blending threshold, in step S720, the image blending module 450takes the pixels of the blur image Img1_blur corresponding to theparameters as pixels of the same positions in the main object clearimage Img1_bokeh, i.e. the pixels of theses positions are in thebackground area, so that the pixels of the blur image Img1_blur areobtained to produce the image with blurry background.

If the parameters are not greater than the first blending threshold, instep S730, the image blending module 450 determines whether theparameters are greater than a second blending threshold. If theparameters are greater than the second blending threshold, in step S740,the image blending module 450 calculates the pixels of the main objectclear image Img1_bokeh corresponding to the parameters according to theparameters. In detail, the positions of the pixels corresponding to theparameters between the first blending threshold and the second blendingthreshold are determined to be located in the edge area connectedbetween the background area and the main object area. The pixels in theedge area connected between the background area and the main object areain the main object clear image Img1_bokeh could be obtained by blendingthe first image Img1 and the blur image Img1_blur.

If the parameters are not greater than the second blending threshold, instep S750, the image blending module 450 obtains pixels of the firstimage Img1 corresponding to the parameters to serve as pixels of themain object clear image Imag1_bokeh. Namely, the positions correspondingto the parameters are determined to be within the main object area, sothat the pixels in the main object area of the first image Img_(—)1 areobtained to serve as the pixels in the main object area of the mainobject clear image Imag1_bokeh, where the first blending threshold isgreater than the second blending threshold.

For example, it is assumed that the image blending module 450 sets theparameter to be between 0 and 255, the image blending module 450 cangenerate the main object clear image Imag1_bokeh by using followingpseudo code (2):

if (Map ≧ Blend_TH1) //Background area (2)  Img1_Bokeh = Img1_Blur elseif (Map ≧ Blend_TH2) //Transition area  w_(Bokeh) = LUT[Map] (LUT istable and value range is 0~255)  ${{Img}\; 1{\_ Bokeh}} = \frac{{w_{Bokeh} \times {Img}\; 1} + {\left( {256 - w_{Bokeh}} \right) \times {Img}\; 1{\_ Blur}}}{256}$else //Subject  Img1_Bokeh = Img1In the present exemplary embodiment, Blend_TH1 is the first blendingthreshold, Blend_TH2 is the second blending threshold, Map is aplurality of parameters in the bokeh map bokeh_map, and LUT[ ] is atable lookup function. It should be noticed that the pixels in the edgearea can be calculated according to a concept of weight. As that shownin the aforementioned exemplary pseudo code, the parameters are taken asa blending weight W_(bokeh), and the pixels in the edge area are blendedaccording to the blending weight W_(bokeh). Namely, regarding a pixel inthe edge area, a blur degree thereof is determined according to whethera position thereof is closer to the main object area or the blur area,and in this way, the main object clear image Img1_bokeh with naturallyconnected main object area and background area is produced, such thatthe edge between the main object and the background in the bokeh imagecan be soft and natural.

In the aforementioned embodiment, the second focal length is, forexample, focused on the background, and a background blur image with ablurry background and clear main object is produced. According to thedescription of FIG. 3, it is known that the image processing method ofthe invention may obtain the final output image according to a pluralityof images. In this way, in other embodiments, when the image capturingdevice captures another image with a third focal length focused on theforeground, the image capturing device can produce an image with blurryforeground and background and clear main object through calculation byusing the aforementioned background blur image and the image captured byfocusing on the foreground according to a process the same with thatused for producing the background blur image.

FIG. 8 is a block diagram of an image capturing device according tostill another embodiment of the invention. Referring to FIG. 8, in thepresent embodiment, the image capturing device 800 is used for producinga full DOF image. The image capturing device 800 includes an imagecapturing module 810, an image calibrating module 820, a gradientcalculating module 830, a map generating module 840, an image blendingmodule 850 and a map adjusting module 860. The image capturing module810, the image calibrating module 820, the gradient calculating module830, the map generating module 840 and the image blending module 850 aresimilar to the image capturing module 410, the image calibrating module420, the gradient calculating module 430, the map generating module 440and the image blending module 450 of FIG. 4, so that details thereof arenot repeated.

It should be noticed that different to the image capturing device 400 ofFIG. 4, the image capturing device 800 of the present embodiment doesnot include the image blurring module but includes a map adjustingmodule 860. The map adjusting module 860 is used for adjusting theparameter map generated by the map generating module 840. In the presentembodiment, the image capturing module 810 captures the first image Img1with the first focal length, and captures the second image Img2 with thesecond focal length, where the first focal length is focused on at leastone main object, and the second focal length is focused on an areaoutside the main object.

Then, the image calibrating module 830 performs the geometriccalibration procedure on the second image Img2 to produce the motioncalibrated second image Img2_cal. The gradient calculating module 830performs a gradient operation on each pixel of the first image Img1 toproduce a plurality of first gradients G1, and performs the gradientoperation on each pixel of the motion calibrated second image Img2_calto produce a plurality of second gradients G2. Then, the map generatingmodule 840 compares each of the first gradients G1 and the correspondingsecond gradient G2 to generate a plurality of comparison results, andgenerates a parameter map according to the comparison results. The stepsthat the image calibrating module 820 generates the motion calibratedsecond image Img2_cal, the steps that the gradient calculating module830 performs the gradient operation, and the steps that the mapgenerating module 840 generates the parameter map are similar to that ofthe image capturing device 400 of FIG. 4, which can be deduced accordingto related descriptions of FIG. 4 and FIG. 5.

Generally, pixels located at a same position in two images havedifferent gradients, i.e. the aforementioned first gradient G1 and thesecond gradient G2. On the other hand, regarding the pixel of the sameposition, if the pixel of such position has a higher gradient in thefirst image (i.e. G1 is greater than G2), it represents that the pixelof such position is located at a clearer area of the first image (i.e.an area within the first focal length). If the pixel of such positionhas a higher gradient in the second image (i.e. G2 is greater than G1),it represents that the pixel of such position is located at a clearerarea of the second image (i.e. an area within the second focal length).Namely, the map generating module 840 can obtain the parameter mapaccording to the pseudo code (1), though the invention is not limitedthereto.

Therefore, in the present embodiment, the map generating module 440generates the parameter map according to comparison results of thegradients of the pixels in the first image Img1 and the motioncalibrated second image Img2_cal. In other words, the parameter mapcarries comparison result information of the gradients of the pixelslocated at the same position in the first image Img1 and the motioncalibrated second image Img2_cal. In this way, the image capturingdevice 800 can learn whether a pixel of a certain position is located ata clear part within the first focal length in the first image Img1 orlocated at a clear part within the second focal length in the secondimage Imge2 according to the parameter map. In this way, the imageblending module 850 can blend the clear parts of the two images forproduce an output image with more clear parts.

It should be noticed that during a process that the user continuouslyshoots a same scene to capture the first image and the second image, dueto a time difference on shooting, individual objects are probably movedin the scene. The image calibrating module 820 performs an overallmotion calibration on the image (or camera motion), and does notcalibrate individual objects in the scene, so that if the image has themoved individual objects, the blended full DOF image may have a ghostphenomenon. The map adjusting module 860 of the present embodiment isused for mitigating the aforementioned ghost phenomenon.

Therefore, the map adjusting module 860 calculates a plurality of sum ofabsolute differences corresponding to each pixel according to a pixelvalue of each of the pixels in the first image Img1 and the second imageImg2, and adjusts the parameters in the parameter map according to thesums of absolute differences. The map adjusting module 860 blends thefirst image Img1 and the motion calibrated second image Img2_calaccording to the adjusted parameter map to generate a full DOF image.

In detail, n×n pixel blocks (n is a positive integer) are first obtainedfrom the first image Img1. It is assumed that n is 5, the obtained 5×5pixel blocks are as that shown in FIG. 9A, which include 25 pixelpositions P₀₀-P₄₄. Similarly, n×n pixel blocks that take the pixelpositions as centers are obtained from the motion calibrated secondimage Img2_cal. Then, sums of absolute differences of specific colorspace components of each pixel in the n×n pixel blocks of the firstimage Img1 and the motion calibrated second image Img2_cal arecalculated, and a representative maximum value thereof is found. Thesums of absolute differences can reflect whether characteristics of thefirst image Img1 and the motion calibrated second image Img2_cal areclose or not in the local area of the n×n pixel blocks. Under a YCbCrcolor space, the specific color space components include a luminancecomponent, a blue chrominance component and a red chrominance component,though the color space is not limited by the invention. Under the YCbCrcolor space, it is assumed that n=5 and the sums of absolute differencesSAD between the pixel positions in the first image Img1 and the motioncalibrated second image Img2_cal are calculated according to followingequations:

${SAD\_ Y} = {\sum\limits_{{i = 0},{j = 0}}^{i = {{4 \cdot j} = 4}}{{{Y\; 1_{ij}} - {Y\; 2_{ij}}}}}$${SAD\_ Cb} = {\sum\limits_{{i = 0},{j = 0}}^{i = {{4 \cdot j} = 4}}{{{{Cb}\; 1_{ij}} - {{Cb}\; 2_{ij}}}}}$${SAD\_ Cr} = {\sum\limits_{{i = 0},{j = 0}}^{i = {{4 \cdot j} = 4}}{{{{Cr}\; 1_{ij}} - {{Cr}\; 2_{ij}}}}}$SAD = max (max (SAD_Y, SAD_Cb), SAD_Cr)Where, i and j represent pixel positions, for example, in the example ofFIG. 9A, each pixel block includes 25 pixel positions P₀₀-P₄₄. Y1_(ij)is a luminance component of a pixel P_(ij) in the first image, andY2_(ij) is a luminance component of the pixel P_(ij) in the secondimage. Cb1_(ij) is a blue chrominance component of the pixel P_(ij) inthe first image, and Cb2_(ij) is a blue chrominance component of thepixel P_(ij) in the second image. Cr1_(ij) is a red chrominancecomponent of the pixel P_(ij) in the first image, and Cr2_(ij) is a bluechrominance component of the pixel P_(ij) in the second image. SAD_Y,SAD_Cb and SAD_Cr are sums of absolute differences on each specificcolor space component.

In this way, the map adjusting module 860 of the present embodiment, forexample, obtain the sums of absolute differences SAD according to theaforementioned equations. Thereafter, the map adjusting module 860determines whether the sums of absolute differences SAD are greater thana motion threshold TH_SAD. If the sums of absolute differences SAD arenot greater than the motion threshold TH_SAD, it represents that thepixel block does not have a phenomenon of captured object movement, sothat it is unnecessary to adjust the parameters in the parameter mapcorresponding to the pixel block. If the sums of absolute differencesSAD are greater than the motion threshold TH_SAD, it represents that thepixel block has the phenomenon of captured object movement, and the mapadjusting module 860 adjusts the parameters in the parameter mapcorresponding to the pixel block according to magnitudes of the sums ofabsolute differences SAD. For example, the map adjusting module 860 cangenerate an adjusted parameter map allin_map according to followingpseudo code (3):

  (3) if (SAD > TH_SAD) Fac = LUT[SAD]; allin_map = map × Fac elseallin_map = mapWhere, Fac represents a weighting factor used by the map adjustingmodule 860 for adjusting the parameter map. Therefore, when the sums ofabsolute differences SAD are greater than the motion threshold TH_SAD,the map adjusting module 860 determines the weighting factor Fac of eachparameter according to the sum of absolute differences SAD, and adjuststhe parameter in the parameter map according to the weighting factorFac. The weighting factor Fac decreases as the sum of absolutedifferences SAD increases.

FIG. 9B is a schematic diagram illustrating a relationship between thesum of absolute differences and the weighting factor according to stillanother embodiment of the invention. Referring to FIG. 9B, when the sumof absolute differences SAD is greater than the motion threshold TH_SAD,the map adjusting module 860 determines the weighting factor of eachparameter according to the sum of absolute differences SAD, and adjuststhe parameter according to the weighting factor. The weighting factordecreases as the sum of absolute differences SAD increases. Namely, eachparameter decreases as the corresponding sum of absolute differences SADincreases.

Then, the image blending module 850 blends the first image Img1 and themotion calibrated second image Img2_cal according to the adjustedparameter map alline_map, so as to produce a full DOF image Img_AIFwithout the ghost phenomenon. The steps that the image blending module850 generates the full DOF image according to the adjusted parameter mapallin_map are similar to the steps that the image blending module 450generates the bokeh image according to the bokeh map bokeh_map, anddetails thereof can be deduced according to related description of FIG.7, which are not repeated. For example, the image blending module 850may obtain the final full DOF image Img_AIF according to followingpseudo code (4):

if (Map ≧ Blend_TH1) //In-of-focus area of image 2 (4)  Img1_AIF = Img2else if (Map ≧ Blend_TH2) //Transition area  w_(AIF) = LUT[Map] (LUT istable and value range is 0~255)  ${{Img}\; 1{\_ AIF}} = \frac{{w_{AIF} \times {Img}\; 1} + {\left( {256 - w_{AIF}} \right) \times {Img}\; 2}}{256}$else //In-of-focus area of image 1  Img1_AIF = Img1Where, in the exemplary pseudo code (4), it is assumed that theparameters are between 0 and 255, Blend_TH1 is the first blendingthreshold, Blend_TH2 is the second blending threshold, Map is aplurality of parameters in the adjusted parameter map allin_map, andLUT[ ] is a table lookup function. It should be noticed that the pixelsin the edge area can be calculated according to a concept of weight. Asthat shown in the aforementioned exemplary program codes, the parametersare taken as a blending weight W_(AIF), and the pixels in the edge areaare blended according to the blending weight W_(AIF).

Similarly, according to related description of FIG. 3, it is known thatthe image processing method of the present embodiment can obtain thefinal output image according to a plurality of images. Therefore, in thepresent embodiment, the image capturing device 800 may capture aplurality of images with a plurality of different focal lengths, andblend the images captured with different focal lengths to produce aclear full DOF image. In an actual application, the scene is firstanalysed to determine the number of images of different focal lengthsthat are required for producing the entirely clear full DOF image.

In summary, according to the image capturing device and the imageprocessing method of the invention, the parameter map is calculated byusing at least two images of different focal lengths, and the mainobject clear image or the full DOF image is generated by blendingaccording to the parameter map. According to the image processing methodof the invention, one or more main objects can be clear and thebackground is blurry, so as to highlight the one or more main objects inthe image. Besides, a connecting edge between the main object and thebackground in the image can be soft and natural, so as to obtain animage with good and natural bokeh effect. On the other hand, the imagescaptured with different focal lengths can be used to construct anentirely clear full DOF image. Moreover, when the full DOF image isconstructed, the noises in the image can also be eliminated, so as toensure that the constructed full DOF image does not lose details of theimage.

It will be apparent to those skilled in the art that variousmodifications and variations can be made to the structure of theinvention without departing from the scope or spirit of the invention.In view of the foregoing, it is intended that the invention covermodifications and variations of this invention provided they fall withinthe scope of the following claims and their equivalents.

What is claimed is:
 1. An image processing method, adapted to an imagecapturing device, the image processing method comprising: capturing afirst image with a first focal length, and capturing a second image witha second focal length, wherein the first focal length is focused on atleast one main object; performing a geometric calibration procedure onthe second image to produce the motion calibrated second image;performing a gradient operation on each pixel of the first image toproduce a plurality of first gradients, and performing the gradientoperation on each pixel of the motion calibrated second image to producea plurality of second gradients; comparing each of the first gradientsand the corresponding second gradient to generate a plurality of firstpixel comparison results, and generating a first parameter map accordingto the first pixel comparison results; and producing a blending imageaccording to the first parameter map and the first image, and producingan output image at least in accordance with the blending image, whereinthe step of comparing each of the first gradients and the correspondingsecond gradient to generate the first pixel comparison results, andgenerating the first parameter map according to the first pixelcomparison results comprises: dividing the second gradients by thecorresponding first gradients to generate a plurality of gradientcomparison values; and generating a plurality of parameters according tothe gradient comparison values, and recording the parameters as thefirst parameter map.
 2. The image processing method as claimed in claim1, wherein the step of producing the output image at least in accordancewith the blending image comprises: capturing a third image with a thirdfocal length; performing the geometric calibration procedure on thethird image to produce the motion calibrated third image; performing thegradient operation on each pixel of the blending image to generate aplurality of third gradients, and performing the gradient operation oneach pixel of the motion calibrated third image to generate a pluralityof fourth gradients; comparing each of the third gradients and thecorresponding fourth gradient to generate a plurality of second pixelcomparison results, and generating a second parameter map according tothe second pixel comparison results; and blending the motion calibratedthird image and the blending image according to the second parameter mapto produce the output image.
 3. The image processing method as claimedin claim 1, wherein the step of performing the geometric calibrationprocedure on the second image to produce the motion calibrated secondimage comprises: performing a motion amount estimation on the firstimage and the second image to calculate a homography matrix; andperforming a geometric affine transformation on the second imageaccording to the homography matrix, so as to obtain the motioncalibrated second image.
 4. The image processing method as claimed inclaim 1, wherein the step of generating the parameters according to thegradient comparison values comprises: determining whether the gradientcomparison values are greater than a first gradient threshold; andsetting the parameters corresponding to the gradient comparison valuesto a first value when the gradient comparison values are greater thanthe first gradient threshold.
 5. The image processing method as claimedin claim 4, wherein the step of generating the parameters according tothe gradient comparison values comprises: determining whether thegradient comparison values are greater than a second gradient thresholdwhen the gradient comparison values are not greater than the firstgradient threshold; setting the parameters corresponding to the gradientcomparison values to a second value when the gradient comparison valuesare greater than the second gradient threshold; and setting theparameters corresponding to the gradient comparison values to a thirdvalue when the gradient comparison values are not greater than thesecond gradient threshold, wherein the first gradient threshold isgreater than the second gradient threshold.
 6. The image processingmethod as claimed in claim 1, wherein the step of producing the blendingimage according to the first parameter map and the first imagecomprises: performing a blur procedure on the first image to generate ablur image; and blending the first image and the blur image according tothe first parameter map to produce a main object clear image.
 7. Theimage processing method as claimed in claim 6, wherein the step ofblending the first image and the blur image according to the firstparameter map to produce the main object clear image comprises:determining whether the parameters are greater than a first blendingthreshold; obtaining pixels of the blur image corresponding to theparameters to serve as pixels of the main object clear image when theparameters are greater than the first blending threshold; determiningwhether the parameters are greater than a second blending threshold whenthe parameters are not greater than the first blending threshold;calculating pixels of the main object clear image according to theparameters when the parameters are greater than the second blendingthreshold; and obtaining pixels of the first image corresponding to theparameters to serve as pixels of the main object clear image when theparameters are not greater than the second blending threshold, whereinthe first blending threshold is greater than the second blendingthreshold.
 8. The image processing method as claimed in claim 1, whereinthe step of producing the blending image according to the firstparameter map and the first image comprises: calculating a plurality ofsums of absolute differences corresponding to each pixel according to apixel value of each of the pixels in the first image and the secondimage, and adjusting the parameters in the first parameter map accordingto the sums of absolute differences; and blending the first image andthe motion calibrated second image according to the adjusted firstparameter map to generate a full depth of field image.
 9. The imageprocessing method as claimed in claim 8, wherein the step of calculatingthe sums of absolute differences corresponding to each pixel accordingto the pixel value of each of the pixels in the first image and thesecond image and adjusting the parameters in the first parameter mapaccording to the sums of absolute differences comprises: determining aweighting factor of each of the parameters according to the sums ofabsolute differences when the sums of absolute differences are greaterthan a motion threshold, and adjusting the parameters according to theweighting factor, wherein each of the parameters decreases as thecorresponding sum of absolute difference increases.
 10. The imageprocessing method as claimed in claim 8, wherein the step of blendingthe first image and the motion calibrated second image according to thefirst parameter map to generate the full depth of field image comprises:determined whether the parameters are greater than a first blendingthreshold; obtaining pixels of the motion calibrated second imagecorresponding to the parameters to serve as pixels of the full depth offield image when the parameters are greater than the first blendingthreshold; determining whether the parameters are greater than a secondblending threshold when the parameters are not greater than the firstblending threshold; calculating pixels of the full depth of field imageaccording to the parameters when the parameters are greater than thesecond blending threshold; and obtaining pixels of the first imagecorresponding to the parameters to serve as pixels of the full depth offield image when the parameters are not greater than the second blendingthreshold, wherein the first blending threshold is greater than thesecond blending threshold.
 11. An image capturing device, comprising: animage capturing module, comprising a zoom lens and a photosensitiveelement, capturing a first image with a first focal length and capturinga second image with a second focal length, wherein the first focallength is focused on at least one main object; and a processor,configured to execute a plurality of modules to: perform a geometriccalibration procedure on the second image to produce the motioncalibrated second image; perform a gradient operation on each pixel ofthe first image to produce a plurality of first gradients, andperforming the gradient operation on each pixel of the motion calibratedsecond image to produce a plurality of second gradients; compare each ofthe first gradients and the corresponding second gradient to generate aplurality of first pixel comparison results, and generating a firstparameter map according to the first pixel comparison results; andproduce a blending image according to the first parameter map and thefirst image, and producing an output image at least in accordance withthe blending image, wherein the processor is configured to execute themodules to divide the second gradients by the corresponding firstgradients to generate a plurality of gradient comparison values, andgenerates a plurality of parameters according to the gradient comparisonvalues, and records the parameters as the first parameter map.
 12. Theimage capturing device as claimed in claim 11, wherein the processor isconfigured to execute the modules to capture a third image with a thirdfocal length, perform the geometric calibration procedure on the thirdimage to produce the motion calibrated third image, perform the gradientoperation on each pixel of the blending image to generate a plurality ofthird gradients, perform the gradient operation on each pixel of themotion calibrated third image to generate a plurality of fourthgradients, compare each of the third gradients and the correspondingfourth gradient to generate a plurality of second pixel comparisonresults, generate a second parameter map according to the second pixelcomparison results, and blend the motion calibrated third image and theblending image according to the second parameter map to produce theoutput image.
 13. The image capturing device as claimed in claim 11,wherein the processor is configured to execute the modules to perform ablur procedure on the first image to generate a blur image, and blendthe first image and the blur image according to the first parameter mapto produce a main object clear image.
 14. The image capturing device asclaimed in claim 11, wherein the processor is configured to execute themodules to calculate a plurality of sums of absolute differencescorresponding to each pixel according to a pixel value of each of thepixels in the first image and the second image, adjust the parameters inthe first parameter map according to the sums of absolute differences,and blend the first image and the motion calibrated second imageaccording to the adjusted first parameter map to generate a full depthof field image.